Skip to content

AMDGPU/GlobalISel: Add regbanklegalize rules for uniform global loads #145909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

AMDGPU/GlobalISel: Add regbanklegalize rules for uniform global loads #145909

wants to merge 1 commit into from
+98 −0

Conversation

petar-avramovic
Copy link
Collaborator

No description provided.

@petar-avramovic Graphite App
Copy link
Collaborator Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

This was referenced Jun 26, 2025
Open
Open
Open
@petar-avramovic petar-avramovic marked this pull request as ready for review June 26, 2025 14:55
@llvmbot
Copy link
Member

llvmbot commented Jun 26, 2025 •
edited
Loading

@llvm/pr-subscribers-llvm-globalisel

@llvm/pr-subscribers-backend-amdgpu

Author: Petar Avramovic (petar-avramovic)

Changes

Full diff: https://github.com/llvm/llvm-project/pull/145909.diff

2 Files Affected:

  • (modified) llvm/lib/Target/AMDGPU/AMDGPURegBankLegalizeRules.cpp (+3)
  • (added) llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll (+95)
diff --git a/llvm/lib/Target/AMDGPU/AMDGPURegBankLegalizeRules.cpp b/llvm/lib/Target/AMDGPU/AMDGPURegBankLegalizeRules.cpp
index a60855cc4f2d6..41d24c8797426 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPURegBankLegalizeRules.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPURegBankLegalizeRules.cpp
@@ -671,6 +671,9 @@ RegBankLegalizeRules::RegBankLegalizeRules(const GCNSubtarget &_ST,
       .Any({{{UniB256, UniP1}, isAlign4 && isUL}, {{SgprB256}, {SgprP1}}})
       .Any({{{UniB512, UniP1}, isAlign4 && isUL}, {{SgprB512}, {SgprP1}}})
       .Any({{{UniB32, UniP1}, !isAlign4 || !isUL}, {{UniInVgprB32}, {SgprP1}}})
+      .Any({{{UniB64, UniP1}, !isAlign4 || !isUL}, {{UniInVgprB64}, {SgprP1}}})
+      .Any({{{UniB96, UniP1}, !isAlign4 || !isUL}, {{UniInVgprB96}, {SgprP1}}})
+      .Any({{{UniB128, UniP1}, !isAlign4 || !isUL}, {{UniInVgprB128}, {SgprP1}}})
       .Any({{{UniB256, UniP1}, !isAlign4 || !isUL}, {{UniInVgprB256}, {VgprP1}, SplitLoad}})
       .Any({{{UniB512, UniP1}, !isAlign4 || !isUL}, {{UniInVgprB512}, {VgprP1}, SplitLoad}})
 
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll
new file mode 100644
index 0000000000000..d7dbde3a2bb00
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll
@@ -0,0 +1,95 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -global-isel -new-reg-bank-select -mtriple=amdgcn-amd-amdpal -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck %s
+
+define amdgpu_ps void @uniform_load_32(ptr addrspace(1) inreg %ptr0, ptr addrspace(1) inreg %ptr1, ptr addrspace(1) inreg %ptr2) {
+; CHECK-LABEL: uniform_load_32:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    v_mov_b32_e32 v0, 0
+; CHECK-NEXT:    global_load_dword v1, v0, s[0:1] glc dlc
+; CHECK-NEXT:    s_waitcnt vmcnt(0)
+; CHECK-NEXT:    global_load_dword v2, v0, s[2:3] glc dlc
+; CHECK-NEXT:    s_waitcnt vmcnt(0)
+; CHECK-NEXT:    v_readfirstlane_b32 s0, v1
+; CHECK-NEXT:    v_readfirstlane_b32 s1, v2
+; CHECK-NEXT:    s_add_i32 s0, s0, s1
+; CHECK-NEXT:    v_mov_b32_e32 v1, s0
+; CHECK-NEXT:    global_store_dword v0, v1, s[4:5]
+; CHECK-NEXT:    s_endpgm
+  %load0 = load volatile i32, ptr addrspace(1) %ptr0
+  %load1 = load volatile i32, ptr addrspace(1) %ptr1
+  %sum = add i32 %load0, %load1
+  store i32 %sum, ptr addrspace(1) %ptr2
+  ret void
+}
+
+define amdgpu_ps void @uniform_load_64(ptr addrspace(1) inreg %ptr0, ptr addrspace(1) inreg %ptr1) {
+; CHECK-LABEL: uniform_load_64:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    v_mov_b32_e32 v2, 0
+; CHECK-NEXT:    global_load_dwordx2 v[0:1], v2, s[0:1] glc dlc
+; CHECK-NEXT:    s_waitcnt vmcnt(0)
+; CHECK-NEXT:    v_readfirstlane_b32 s0, v0
+; CHECK-NEXT:    v_readfirstlane_b32 s1, v1
+; CHECK-NEXT:    s_add_i32 s0, s0, s1
+; CHECK-NEXT:    v_mov_b32_e32 v0, s0
+; CHECK-NEXT:    global_store_dword v2, v0, s[2:3]
+; CHECK-NEXT:    s_endpgm
+  %load = load volatile <2 x i32>, ptr addrspace(1) %ptr0
+  %elt0 = extractelement <2 x i32> %load, i32 0
+  %elt1 = extractelement <2 x i32> %load, i32 1
+  %sum = add i32 %elt0, %elt1
+  store i32 %sum, ptr addrspace(1) %ptr1
+  ret void
+}
+
+define amdgpu_ps void @uniform_load_96(ptr addrspace(1) inreg %ptr0, ptr addrspace(1) inreg %ptr1) {
+; CHECK-LABEL: uniform_load_96:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    v_mov_b32_e32 v3, 0
+; CHECK-NEXT:    global_load_dwordx3 v[0:2], v3, s[0:1] glc dlc
+; CHECK-NEXT:    s_waitcnt vmcnt(0)
+; CHECK-NEXT:    v_readfirstlane_b32 s0, v0
+; CHECK-NEXT:    v_readfirstlane_b32 s1, v1
+; CHECK-NEXT:    v_readfirstlane_b32 s4, v2
+; CHECK-NEXT:    s_add_i32 s0, s0, s1
+; CHECK-NEXT:    s_add_i32 s0, s0, s4
+; CHECK-NEXT:    v_mov_b32_e32 v0, s0
+; CHECK-NEXT:    global_store_dword v3, v0, s[2:3]
+; CHECK-NEXT:    s_endpgm
+  %load = load volatile <3 x i32>, ptr addrspace(1) %ptr0
+  %elt0 = extractelement <3 x i32> %load, i32 0
+  %elt1 = extractelement <3 x i32> %load, i32 1
+  %elt2 = extractelement <3 x i32> %load, i32 2
+  %sum0 = add i32 %elt0, %elt1
+  %sum = add i32 %sum0, %elt2
+  store i32 %sum, ptr addrspace(1) %ptr1
+  ret void
+}
+
+define amdgpu_ps void @uniform_load_128(ptr addrspace(1) inreg %ptr0, ptr addrspace(1) inreg %ptr1) {
+; CHECK-LABEL: uniform_load_128:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    v_mov_b32_e32 v4, 0
+; CHECK-NEXT:    global_load_dwordx4 v[0:3], v4, s[0:1] glc dlc
+; CHECK-NEXT:    s_waitcnt vmcnt(0)
+; CHECK-NEXT:    v_readfirstlane_b32 s0, v0
+; CHECK-NEXT:    v_readfirstlane_b32 s1, v1
+; CHECK-NEXT:    v_readfirstlane_b32 s4, v2
+; CHECK-NEXT:    v_readfirstlane_b32 s5, v3
+; CHECK-NEXT:    s_add_i32 s0, s0, s1
+; CHECK-NEXT:    s_add_i32 s0, s0, s4
+; CHECK-NEXT:    s_add_i32 s0, s0, s5
+; CHECK-NEXT:    v_mov_b32_e32 v0, s0
+; CHECK-NEXT:    global_store_dword v4, v0, s[2:3]
+; CHECK-NEXT:    s_endpgm
+  %load = load volatile <4 x i32>, ptr addrspace(1) %ptr0
+  %elt0 = extractelement <4 x i32> %load, i32 0
+  %elt1 = extractelement <4 x i32> %load, i32 1
+  %elt2 = extractelement <4 x i32> %load, i32 2
+  %elt3 = extractelement <4 x i32> %load, i32 3
+  %sum0 = add i32 %elt0, %elt1
+  %sum1 = add i32 %sum0, %elt2
+  %sum = add i32 %sum1, %elt3
+  store i32 %sum, ptr addrspace(1) %ptr1
+  ret void
+}

arsenm
arsenm reviewed Jun 27, 2025
View reviewed changes
Copy link
Contributor

@arsenm arsenm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with test nits

llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll Outdated
@@ -0,0 +1,95 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
; RUN: llc -global-isel -new-reg-bank-select -mtriple=amdgcn-amd-amdpal -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck %s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
; RUN: llc -global-isel -new-reg-bank-select -mtriple=amdgcn-amd-amdpal -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck %s
; RUN: llc -global-isel -new-reg-bank-select -mtriple=amdgcn-amd-amdpal -mcpu=gfx1010 < %s | FileCheck %s

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove verify-machineinstrs?

llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll Outdated
@@ -0,0 +1,95 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test name could be more specific, it's uniform VGPR loads.

llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll Outdated
store i32 %sum, ptr addrspace(1) %ptr2
ret void
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth testing other cases with other element types? ptr, i16 vectors, fp, etc.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check is for size in bits not specific type.
Maybe later when we go over all possible load types? I wanted to get readanylane combines out of the way first.

arsenm
arsenm reviewed Jun 27, 2025
View reviewed changes
llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform.ll Outdated
%sum = add i32 %sum1, %elt3
store i32 %sum, ptr addrspace(1) %ptr1
ret void
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alignment check not stressed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this one. Align condition is from old regbankselect but maybe we don't need it, or it is wrong?
Legalizer deals with many cases of Align and splits load.
Do you know what we need to check for considering align?

@petar-avramovic petar-avramovic force-pushed the users/petar-avramovic/rbl-uniform-loads branch from 3a23ac4 to c6461ff Compare June 27, 2025 10:39
@petar-avramovic petar-avramovic force-pushed the users/petar-avramovic/rbl-uniform-loads branch from c6461ff to dea46d7 Compare June 27, 2025 13:52
petar-avramovic
petar-avramovic commented Jun 27, 2025
View reviewed changes
llvm/test/CodeGen/AMDGPU/GlobalISel/load-uniform-in-vgpr.ll
; CHECK-NEXT: global_store_dword v0, v1, s[4:5]
; CHECK-NEXT: s_endpgm
%load0 = load volatile i32, ptr addrspace(1) %ptr0
%load1 = load i32, ptr addrspace(1) %ptr1, align 1
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated with cases that hit align check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arsenm arsenm arsenm left review comments

@nhaehnle nhaehnle nhaehnle approved these changes

@Pierre-vh Pierre-vh Pierre-vh approved these changes

Assignees
No one assigned
Labels
backend:AMDGPU llvm:globalisel
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@petar-avramovic @llvmbot @arsenm @nhaehnle @Pierre-vh